THREE-DIMENSIONAL VISUAL INSPECTION METHOD OF 
SEMICONDUCTOR PACKAGES AND APPARATUS USING SINGLE 
CAMERA 



BACKGROUND OF THE INVENTION 



Field of the Invention 



The present invention relates to a three-dimensional 
visual inspection method of semiconductor packages and 
apparatus thereof, and more particularly relates to a 
three-dimensional visual inspection method of 
semiconductor packages and apparatus using single 
camera, which is able to carry out a three-dimensional 
visual inspection of small-size, high-density 
semiconductor packages having a large-scale of 
integration by using a single camera. 



Description of the Related Art 



Recently, as the integration scale of semiconductor 
element becomes to be large and the size of element 
rapidly reduces, the production of semiconductor 
element using an appropriate package style for a high- 
density element, such as a ball grid array(BGA) 



package, a plastic quad flat package ( PQFP ) , or a small 
outline package (SOP), increases. 

Here, a BGA package is formed by adhering an element 
directly to a through-hole of an electronic circuit 
5 board by heating a circular-shape lead placed at the 

bottom of the element, and an SOP is achieved by 
densely forming numbers of very-fine legs on all sides 
13 of the package and the element. 

□ In case of the packages having a style of PQFP or SOP 

■10 above mentioned, neighboring legs had been arranged 

^ generally with the space of larger than 1.0 mm in the 

past, however the space between legs is recently 
™ reduced to under 0.2 mm, which can be hardly 

: recognized by human eye. Therefore, a three- 

15 dimensional inspection becomes to be required. 

Because the above-described packages are assembled by 

being pressed with heat at the top or bottom of an 

electronic circuit board, if the heights of balls 

adhered to the bottom of the element is not uniform or 
20 the heights of the legs formed an the sides of the 

element are not uniform, the whole board is not usable 

due to contact failure. 
To solve these problems, an inspection method to 

check whether the heights of the legs or the balls of 
25 the element are uniform or not is required. 
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The three-dimensional inspection methods used in the 
prior art employ a method using two cameras or a 
method using a camera and a laser source of which the 
interrelationship is known. They obtain the distances 
from camera to the balls (or legs), construct a plane 
by these data, and thereby check whether the height is 
uniform or no t . 

Here, the method using two cameras has disadvantages 
that, in case that the size of the element becomes 
small, the installation of cameras is difficult and 
the measurement accuracy becomes low. 

And, the method using a camera and a laser source has 
disadvantages that the price of the equipment becomes 
high in order to construct a laser source, which 
provides a thin laser light having a thickness less 
than 0.1 mm, and it is difficult to obtain the 
interrelationship between the camera and the laser 
source . 



SUMMARY OF THE INVENTION 



The present invention is proposed to solve the 
problems of the prior art mentioned above. It is 
therefore the object of the present invention to 
provide a three-dimensional visual inspection method 



of semiconductor packages and apparatus thereof, which 
carries out a three-dimensional visual inspection by 
using a single camera without using an additional 
special light source like a laser light source and 
lowers the system price to be less than 50% of that of 
the prior three-dimensional visual inspection 
apparatus thereby. 

To achieve the object mentioned above, the present 
invention presents a visual inspection method of 
10 semiconductor packages usrng a single camera, which is 

able to carry out a three-dimensional visual 
inspection that has been recently required as the 
integration scale of the semiconductor element becomes 
large and the size of the element becomes small. 
15 in more detail, the present invention presents a 

three-dimensional visual inspection method that is 
applicable to measure the heignt of the balls in a 
ball grid array(BGA) package which is formed by 
adhering an element directly to a through-hole of an 
20 electronic circuit board by heating a c i rcul ar - shape 

lead placed at the bottom of the element, or to 
measure the height of the legs in a small outline 
package (TSOP or TTSOP) which is formed by densely 
attaching numbers of very-fine legs on all sides of 
25 the element. 



BRIEF DESCRIPTION OF THE DRAWINGS 



FIG. 1 is a view illustrating the structure of a 
5 three-dimensional visual inspection method in 

accordance with the present invention. 

FIG. 2 is a perspective view of a prism in accordance 
with the present invention. 

FIG. 3 is a view illustrating the concept of imaging 
Ho principle using a prism in accordance with the 

present invention. 

FIG. 4 is a view illustrating the field of view(FOV) 
of a real camera using a prism in accordance with the 
present invention. 
U5 FIG. 5 is a view illustrating the field of view(FOV) 

of a virtual stereo camera using a prism in accordance 
with the present invention. 

FIG. 6 is a sample image of a micro-BGA package 
element obtained without using a prism in accordance 
20 with the present invention. 

FIG. 7 is a sample image of a micro-BGA package 
element obtained by using a prism in accordance with 
the present invention. 
FIG. 8 is a flow chart illustrating the image 
25 processing procedures in accordance with the present 
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invention . 

< Description of the Numerals on the Main Parts of 
the Drawings> 

10 : an image processing system 

12 : a camera 

14 : a prism 

16 : a lighting means 

18 : a package element 

20 : an image plane 

22 : a camera lens 

DETAILED DESCRIPTION OF THE EMBODIMENTS 



Hereinafter, referring to appended drawings (FIG. 1 ~ 
: 1 ! 5 FIG. 8), the structure and the operation procedures of 

the embodiments of the present invention are described 
in detail. 

The basic concept of a three-dimensional visual 
inspection method in accordance with the present 
20 invention is illustrated in FIG. 1. 

A lighting means (16) is located over the package 
element (18) to be inspected to lighten the element, a 
camera (12) is installea over the lighting means (16), 
and a prism(14) is inserted between the camera (12) and 
25 the package element (18). 



The stereo images obtained through the prism(14) are 
read by a camera (12), input to the image processing 
system(lO), and processed therein for three- 
dimensional visual inspection. 
5 In other words, cop 1 anar i t i e s of main characteristic 

points of the element are measured and inspected by 
measuring the distances from the camera (12) to the 
corresponding points. 

The image processing system(lO) is constructed to be 
-10 a PC-based system or an embedded system. 

= The prism(14) is made of a transparent material, such 

" as a glass or a crystal, and rt has a trigonal shape. 

A light emitting diode (LED) is a desirable lighting 
means (16). 

=15 Since an LED has an irregular reflection 

characteristic in itself, it eliminates the 
inhomogeneous reflection light partially reflected at 
the surface of the subject, such as a highly- 
reflecting metal surface, and obtains a stable image 

20 thereby. 

In addition, since the ball in BGA package has a 
spherical shape, rf a ring-type light is applied from 
the lower position, a doughnut-shaped image of which 
the circumference is bright and the center is dark can 

25 be obtained. Therefore, a ring-type LED is desirable 



for inspecting a BGA package since the image 
processing to extract the vertex of the ball is easy. 

The optical path of the light from the package 
element (18) transmitted through the prism(14) in FIG. 
1 is split into two directions, and thereby a single 
spatial point is mapped in two different points on the 
image plane (20). As a result, a three-dimensional 
inspection can be carried out since a stereo image can 
be obtained by a single camera. 

In the prior art which obtains a stereo image using 
two cameras, the cha r a c t e r i s t i c s ( f oca 1 length, 
exposure, zoom, etc.) of the lenses equipped in two 
cameras are different from each other. And it is very 
difficult to mechanically fix the two cameras for 
their optical axes to be parallel. (In reality, it is 
impossible.) Therefore, it requires a complicated 
algorithm to process the image for compensating these 
defects . 

On the other hand, in case of using single camera (12) 
and a prism(14) like the present invention, the 
problems described above are not occurred since the 
image is read through a single camera lens (22) . 

In particular, a process of calculating an epipolar 
line to find the corresponding points on the stereo 
image is eliminated and the images existing on the 



same horizontal line are to be analyzed. 

Therefore, the whole system efficiency is improved as 
the image processing becomes to be simple and the 
image processing speed becomes to be fast. 
5 FIG. 2 is a perspective view of a prism in accordance 

with the present invention, and FIG. 3 is a view 
illustrating the concept of imaging principle by which 
a single spatial point is mapped into two different 
points on the image plane. 

10 As described in FIG. 3, a single spatial point, XP, 

is converted to two different points, XR and XL, by 

" the prism(14), and thereafter mapped into two 

different points, mR and mL, on the image plane (20) by 
the camera lens (22 ) . 

=15 Here, all the existing spatial points are not 

converted through these processes but only the images 
existing within the narrow range decided by the 
internal angle (a) of the prism(14) are converted. 
The range is illustrated in detail in FIG. 4 and FIG. 

20 5 . 

FIG. 4 is a view illustrating the field of view(FOV) 
of a real camera using a prism in accordance with the 
present invention, and FIG. 5 is a view illustrating 
the FOV of a virtual stereo camera. 
25 In the figures, the FOV is basically divided into 
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three different regions: <- a region both cameras can 
observe, T a region the left-side camera can only 
observe, — >• a region the right-side camera can only 
observe . 

5 Among the regions, the region in which a single 

spatial point is mapped into two different points on 
the image plane is region <— . 

^ Therefore, the subject for three-dimensional 

inspection should be placed in region <-, and then two 

ho different images can be obtained by an image 

i processing based on the imaging principle described in 

^ FIG. 3. 

" The sample images of micro-BGA package elements 

obtained by visual systems are illustrated in FIG. 6 
=45 and FIG . 7 . 

FIG. 6 shows an image obtained without using a prism 
and FIG. 7 shows two images obtained by using a prism 
in accordance with the present invention. 
The image processing procedures to obtain three- 
20 dimensional information using the two images are shown 

in FIG. 8. 

Before starting a visual inspection, 

calibration (S100 ) of a camera is performed to obtain 
the intrinsic parameters ( focal length, scale factor, 
25 distance between the prism(14) and the image plane (20), 
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camera constant, etc.) of the camera (12) and the 
prism(14) by using an object of which the exact three- 
dimensional information is known. 
Next, reading (S102 ) the two images obtained by using 
5 the prism(14), ex t r a c t ing ( S 1 0 4 ) the characteristic 

points, which are corresponding to each other, from 
the two images, ca 1 cu 1 a t i ng ( S i 0 6 ) the disparity 
between two points, the system ex t r a c t s ( S 1 0 8 ) the 

.,: distances to the corresponding points and the three- 

CPo dimensional coordinates therefrom. 

Ul The characteristic points on the image, in the step 

01 of S104, can vary for each application field. In the 

C case of a BGA package, the vertexes of spherical- 

fl shaped balls are used for characteristic points on the 

rf5 image since the image of a ball is mapped into a 

doughnut-shape as illustrated in FIG. 6 and FIG. 7. 
And in the case of an SOP element, the edges at the 
ends of the legs are used for characteristic points on 
the image since the end of the leg has a rectangular 
20 shape. 

Using the three-dimensional information extracted 
through the step, S108, a spatial plane is 
presumed (S110 ) . Thereafter, a planarity 

inspect ion ( SI 12 ) , which is a three-dimensional 
25 inspection, is carried out by analyzing the relative 



distribution of the characteristic points to the plane. 

In other words, if the characteristic points are 
located on the presumed plane, it is considered that 
most characteristic points are located on the same 
plane, and the element is estimated as good. On the 
other hand, if the distance between the point and the 
plane is larger than the prescribed standard value, 
the element is estimated as bad. 

For example, rf the height of the ball of a BGA 
package or the height of the leg of an SOP element 
remains within a prescribed error bound so that the 
assembling process on a planar PCB can be carried out 
with no trouble, the element is estimated as good. And 
in the opposite cases, it is estimated as bad since 
some portion of the PCB contacts too tight and the 
other portion could have contact failure even though 
the PCB is assembled. 

Since a certain pattern is repeated in the case of a 
semiconductor package, the extraction of 

characteristic points on the image in the step, S104, 
and the step, S106, is performed by following an image 
processing algorithm specified for its application 
purpose . 

The three-dimensional distance according to the image 
disparity is calculated by the following equation: 



[Equation 1] 

1 k x . 

— = ^- + k- > 
d Z F 



where, * t =*2*'z, ' 2.a u *ianS f " c x 

Here, d is disparity calculated on the image, [pixel], 
5 ZP is the distance to the characteristic point, [mm], 

kl, k2 are intrinsic parameters of the camera (12), tZ 
is the distance from the image plane (20) to the 
prism(14), [mm], 8 is the internal angle of the 
prism (14), [radian], f is the focal length of the 
ll.O camera lens (22), [mm], and cx is the length of an 

image sensor cell along with X-axis, [mm] . 
iU The values of kl, k2 , tZ, f are decided by 

calibration of camera (12) at step S100. 
5 is decided with respect to FOV according to the 
15 size of a subject, and cx is decided by the size of 

the image array and the resolution of the image after 
the camera(12) is selected. 

After the three-dimensional distances to the main 
characteristic points are extracted by the procedures 
20 described above, the three-dimensional coordinates of 

the corresponding points are calculated by 
t r igonomet r y . 

In other words, the three-dimensional information on 
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the three-dimensional coordinate system, which has its 
origin at the center of the camera lens (22), is 
totally decided and the information on the n 
characteristic points, described below, are obtained: 
5 (x,,y,,z,X i = \,2,---,n, n>A_ 

Here, the accuracy of the information on the three- 
dimensional distance can be improved by linearly 
interpolating the changes in brightness between 
neighboring pixels and using the quatized data up to 

;;i () the necessary number. 

In other words, analyzing the resolution of the image 

fr| up to the range less than a pixel and improving the 

resolution of the image around the characteristic 
points thereby, the system provides an information on 

C|5 the three-dimensional distance with an improved 

accuracy . 

Next, by using a least-square method or a Hough 
transform, an planar equation, as described in 
Equation 2, is e xt r ac t ed ( S 1 1 0 ) , and the distribution 
20 characteristics of the characteristic points to the 

plane is analy zed ( S 1 1 2 ) . In other words, the 
coplanarity is inspected. 
[Equation 2] 

a*x+b»y+c»z=d 

25 Here, a, b, c, and d are coefficients of the planar 



equation extracted. 

Finally, if the distance between the characteristic 
point and the extracted plane is larger than the 
standard value, the package is estimated as bad. Or, 
5 if the distance exists within a prescribed error bound, 

the package is estimated as good. 

As mentioned thereinbefore, the present invention 
provides a three-dimensional visual inspection method 
7 1 0 of semiconductor packages using single camera. 

For example, the three-dimensional visual inspection 
s=i method in accordance with the present invention is 

f=i applicable to measure the heights of the balls in a 

ball grid array (BGA) package which is formed by 
r|5 adhering an element directly to a through-hole of an 

electronic circuit board by heating a circular-shape 
lead placed at the bottom of the element, or to 
measure the heights of the legs in a small outline 
package (TSOP or TTSOP) which is formed by densely 
20 attaching numbers of very-fine legs on all sides of 

the element . 

In addition, since the inspection method in 
accordance with the present invention carries out a 
three-dimensional visual inspection by using a single 
25 camera without using an additional special light 



